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Abstract 

This  paper  will  provide  a  brief  summary  of  recent  research  at  the 
Air  Force  Institute  of  Technology  (AFIT)  in  the  area  of  Neural 
Networks.  Specifically,  AFIT  research  into  the  areas  of  back- 
propagation  relations  to  conventional  algorithms,  feature  selec¬ 
tion,  neural  networks  for  segmentation  or  general  purpose  pre¬ 
processing,  angle  of  arrival  detection,  speech  recognition,  optical 
neural  networks  and  brain  implants  will  be  presented. 


1  Introduction 

Over  the  last  several  years,  AFIT  has  been  investigating  neural 
networks  for  military  applications.  During  1989,  the  research 
group  consisted  of  masters,  PhD  students  and  faculty  totaling 
approximately  twenty  people.  This  paper  covers  some  of  the 
work  from  this  group. 

2  Back  Propagation  as  a  Degenerate 
Kalman  Filter 

Several  major  accomplishments  were  made  in  the  last  six  months. 
We  were  able  to  show  that  the  most  common  learning  algorithm, 
backpropagation,  used  for  feedforward  artificial  neural  networks 
is  just  a  degenerate  version  of  an  extended  Kalman  filter.  In 
essence  the  result  shows  that  with  certain  degenerate  assump¬ 
tions  the  extended  Kalman  algorthm  becomes  the  backpropaga¬ 
tion  algorithm.  This  result  allows  us  to  call  upon  AFIT’s  rich 
experience  in  Kalman  Filtering  to  determine  the  applicability  of 
these  networks.  On  the  other  hand,  if  the  degenerate  assump¬ 
tions  were  not  made  and  the  full  extended  Kalman  filter  was 
used  to  train  the  weights,  how  are  the  learning  results  affected? 
Figure  1  shows  the  results  of  training  networks  using  both  con¬ 
ventional  backpropagation  and  the  extended  Kalman  filter  on  a 
mesh  problem  with  interconnected,  disjoint  regions.  One  sees 
that  the  Kalman  filter  learns  with  fewer  iterations.  If  the  com¬ 
parison  were  made  on  the  number  of  computations,  the  Kalman 
algorithm  is  far  more  intensive  than  the  conventional  backprop¬ 
agation.  When  more  realistic  problems  in  target  recognition  are 
tested,  the  advantages  of  learning  are  not  as  dramatic.  Figure  2 
shows  both  algorithms  learning  from  absolute  range  data  and  ac¬ 
curacies  are  similar  for  a  given  number  of  iterations.  This  result 
emphasizes  the  importance  of  using  real  data  when  developing 
new  learning  algorithms. 
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3  Conventional 

Statistical  Techniques  versus  Neural 
Based  Classifiers 

A  second  major  accomplishment  was  the  comparison  of  the  re¬ 
sults  of  using  artificial  neural  networks  to  classify  tactical  tar¬ 
get  data  with  conventional  classification  techniques.  The  major 
result  here  is  that  the  networks  can  do  about  as  well  as  the 
optimum  statistical  classification  technique.  The  implication  of 
this  is  that  we  will  not  base  the  use  or  non-use  of  a  neural  net¬ 
work  classifier  on  desired  accuracy  but  potentially  on  other  fac¬ 
tors  such  as  cost,  robustness  or  even  speed.  In  Table  1  we  see 
the  comparison  of  a  conventional  non-par ametric  Bayesian  based 
classifier  and  a  neural  network  and  the  accuracies  show  no  statis¬ 
tically  meaningful  differences.  A  fundamental  mathematical  tie 
between  these  approaches  is  now  being  investigated,  analogous 
to  the  extended  Kalman  filter  backpropagation  tie. 

4  Feature  Selection 

A  third  major  accomplishment  of  the  research  group  was  the 
determination  of  a  method  of  finding  the  importance  of  features 
for  use  by  a  neural  network  classifier.  Specifically,  in  a  tactical 
target  problem  we  submitted  a  set  of  features  and  trained  the 
network  to  tell  the  difference  between  the  different  targets  and 
non-target  blobs  found  by  our  automatic  segmentor.  Using  our 
method,  we  could  determine  which  of  the  features  were  impor¬ 
tant  in  the  discrimination  of  the  classes.  The  importance  here 
is  that  w?e  no  longer  have  to  know  a  priori  what  is  the  right  set 
of  features  but  we  allow  the  network  to  determine  it  from  every 
feature  we  as  the  system  designer  think  might  be  relevant.  We 
have  also  demonstrated  fusing  infomation  from  multiple  sensors. 
In  fact,  we  have  demonstrated  neural  networks  for  fusing  infor¬ 
mation  from  multiple  sensors  increasing  probabilities  of  detec¬ 
tion  without  significantly  increasing  the  false  alarm  rates.  These 
ideas  of  having  an  automatic  system  for  architecting  networks, 
selecting  features  and  fusing  information  for  a  given  problem  will 
allow  this  technology  to  be  used  by  non-neural  network  people 
to  solve  their  problems. 

5  Segmentation 

We  also  developed  techniques  for  using  neural  networks  for  im¬ 
age  segmentation.  To  accomplish  this,  we  used  two  techniques. 
First  we  tested  the  use  of  Hough  transforms  on  discrimination 
of  SAR  images  of  objects.  The  biological  connection  here  is  the 
postulated  existence  of  line  detectors  in  the  primary  visual  cor¬ 
tex  (Hubei  and  Wiesel).  We  believe  that  a  better  model  is  that 
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of  the  Gabor  transform  and  a  technique  to  segment  FLIR  images 
using  Gabor  transforms  was  developed.  The  important  results 
here  are  that  both  of  these  techniques  can  be  implemented  as 
neural  networks  as  front  ends  to  subsequent  neural  network  or 
conventional  classifiers.  This  has  significant  long  term  implica¬ 
tions  since  one  of  the  great  unsolved  problems  in  science  and 
engineering  is  how  do  animals  interact  with  the  environment. 
To  do  this  the  original  noisy,  analog,  real-world  data  must  be 
processed.  The  great  hope  for  neural  networks  is  that  they  can 
handle  this  problem.  Our  contribution  to  using  neural  networks 
(Gabor  transforms)  to  find  targets  in  real  world  images  (segmen¬ 
tation)  demonstrates  that  artificial  neural  networks  can  provide 
a  means  for  processing  real  world  data.  Figure  3  below  shows  an 
original  FLIR  scene  and  the  segmentation  achieved  by  comput¬ 
ing  the  Gabor  transform  of  the  image  and  adding  together  the 
resulting  images  for  four  orientations  of  the  Gabor  wavelets. 

6  Optical  Pattern  Recognition 

In  the  optical  pattern  recognition  world  we  developed  techniques 
which  allow  the  processing  of  real  FLIR  images  with  existing  bi¬ 
nary  spatial  light  modulators.  A  joint  transform  correlator  was 
designed  and  tested  using  images  of  tactical  targets.  The  bina- 
rization  techniques  were  based  on  biological  information  on  nor¬ 
malization  in  the  Limulus  eye.  This  is  a  demonstration  of  optical 
processing  techniques  that  function  with  real  world  FLIR  scenes 
with  existing  optical  components.  The  techniques  developed  in 
this  area  allow  us  now  to  envision  smart  bombs  with  shape  recog¬ 
nition  being  accomplished  at  photon  speed  instead  of  crunching 
large  arrays  of  numbers  in  an  embedded  silicon  computer  in  the 
front  of  a  missile.  The  key  to  success  for  this  correlator  was  the 
neually  inspired  binarization  which  allowed  the  display  of  the 
FLIR  images  on  a  binary  spatial  light  modulator,  a  magneto¬ 
optic  device. 

7  Angle  of  Arrival  Detection 

Also  in  the  area  of  optical  processing,  an  optical  direction  of 
arrival  detector,  applicable  to  laser  illumination  direction  deter¬ 
mination,  was  designed  and  tested.  The  design  is  similar  to  a 
fly’s  eye.  The  intensity  patterns  from  a  fibre  optic  waveguide 
are  processed  to  determine  direction  information.  We  proposed 
the  fly’s  eye  designed,  built  it,  tested  it  and  are  now  refining  the 
neural  network  processing  techniques  in  use.  We  obtained  a  one 
degree  accuracy  on  the  original  prototype  system. 

8  Optical  Operational  Amplifier 

Newly  coated  mirrors  for  the  optical  confocal  Fabry  Perot  in¬ 
terferometer  were  designed,  specified,  fabricated  and  have  now 
been  delivered  and  installed  on  special  mounts  which  we  can 
electronically  control.  We  spent  the  last  six  months  character¬ 
izing  the  beam  fanning  phenomena.  With  this  information  and 
the  newly  fabricated  mirrors  we  are  now  experimenting  with  the 
Optical  Resonator  as  a  testbed  for  a  general  purpose  optical  com¬ 
puter.  Specifically  we  will  test  our  design  as  an  optical  neural 
network  (associative  memory,  as  a  clutter  rejector,  and  an  op¬ 
tical  phase  retrieval  system).  We  have  now  demonstrated  the 
optical  computer  stabilized  and  functioning  and  are  in  the  pro¬ 
cess  of  inserting  the  nonlinear  crystals  which  are  necessary  to  do 
the  applications  listed  above. 


9  Speech  Recognition  and  Processing 

In  the  area  of  speech  recognition  we  made  significant  progress  in 
the  use  of  neural  networks  for  processing  multiple  feature  sets  for 
speech  recognition.  Although  the  accuracies  are  only  comparable 
to  other  existing  techniques  the  fact  that  the  networks  can  be 
competitive  here  gives  us  hope  that  we  now  have  alternatives. 

Also  in  the  area  of  speech  processing,  working  with  muti¬ 
lated  speech  from  Jim  Cupples,  RADC/IRRA,  we  have  devel¬ 
oped  techniques  to  reconstruct  speech  using  rules,  rather  than 
filters.  We  are  now  extending  this  work  to  see  if  it  can  be  the 
basis  of  an  effective  speech  recognizer  and  a  low  bit  speech  en¬ 
coder. 

10  Reverse  VLSI  Engineering 

In  the  area  of  Reverse  VLSI  we  developed  techniques  using  the 
neural  inspired  Gabor  Transform  to  find  landmarks  of  interest 
such  as  the  pads.  As  part  of  this  effort  we  received  and  integrated 
a  Sun  4,  a  digitizer,  an  optics  bench  and  a  laser  microscope.  This 
work  is  documented  in  a  companion  NAECON  paper.  This  is  a 
special  case  of  the  problem  of  using  neural  networks  as  front  end 
processors  for  segmentation. 

11  Multiplexed  Brain  Electrode 

Lastly  we  are  continuing  work  on  a  brain  chip.  The  problem  for 
the  last  couple  of  years  is  how  to  take  commercially  available 
chip  fabrication  technology,  which  uses  aluminum,  find  replate 
these  chips  using  platinum  for  implantation  within  a  living  mon¬ 
key.  We  believe  that  this  problem  is  now  solved  and  hope  for 
implantation  this  year. 
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Figure  1:  Average  Training  Accuracy  for  Exclusive-OR  Problem  -  90  confidence  (solid:  Kalman  training;  dotted:  back  propagation 
training); 


Table  1:  Overall  FLIR  Classification  Accuracy  (95%  Confidence  Interval) 


Classifier 

Accuracy  Rate 

Neural  Network 

(0.883,0.931) 

Bayesian 

(0.841,0.897) 

fTERATONS 


Figure  2:  Average  Testing  Accuracy  for  Target/Non-target  Problem  on  Absolute  Range  Data  -  90%  confidence  (solid:  back  propaga¬ 
tion;  dotted:  Kalman). 
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(a)  Original  FLIR  Image 


(b)  Gabor  Segmented  FLIR  Image 


Figure  3:  Artificial  Neural  Network  Segmentation 
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